Parent and Teacher Ratings of Peer Interactive Play and Social-Emotional Development of Preschool Children at Risk

Estimates of agreement and consistency between parent and teacher ratings of peer-interactive play and social emotional development were examined in a sample of preschool children at risk for academic difficulties. Eighty-three pairs of parents and teachers of 4-year-old children completed the Penn Interactive Peer Play Scale (PIPPS) and 69 of these pairs also were interviewed with the Vineland Social-Emotional Early Childhood Scales (SEEC). Significant mean differences were found between parent and teacher scores on the PIPPS, but not on the SEEC. No noteworthy correlations were found between raters on the PIPPS, though low to moderate correlations were revealed between parent and teacher responses on the SEEC. Multivariate techniques revealed statistically significant and noteworthy correlations between parent and teacher responses on the SEEC but not on the PIPPS. Results are discussed within the context of conducting multifaceted, ecological assessments with preschool children. Implications for early childhood educators and interventionists are noted regarding the importance of gathering and interpreting multiple sources of information that contribute to an understanding of young children’s behavior. The increased emphasis on prevention and early intervention in the context of the Individuals with Disabilities Education Act (IDEA, 2004) and in other early childhood initiatives such as Good Start, Grow Smart (U.S. Department of Health and Human Services, 2005) has led to a rapid increase in the assessment of preschoolers’ social-emotional behavior. The identification of social and emotional problems in young children is critical to intervening early to reduce the likelihood of placement in special education, residential treatment, and incarceration later in life (Squires, Bricker, Heo, & Twombly, 2001). A hallmark of effective assessment is that it is multidimensional, which involves including information from multiple perspectives, instruments, methods, and settings (Bagnato & Neisworth, 1991; Merrell, 1996). Because preschoolers’ behavior is so variable within testing situations (Nagle, 2000), evaluators must rely on input from parents and teachers to obtain complete descriptions of children’s functioning across people and contexts. Obtaining information from both parents and teachers also improves validity. For example, a longitudinal study found that the combination of parent and teacher ratings of social-emotional and behavioral functioning resulted in improved predictive power than rating scale data obtained from only parents or only teachers for children’s academic, behavioral, and mental health outcomes 6 years later (Verhulst, Koot, & Van der Ende, 1994). Preschool Play-Based Assessment A variety of assessment techniques are used to assess preschool social-emotional functioning, including interviews, observations, and third-party rating instruments (Keith & Campbell, 2000; Martin, 1986). To provide the most valid estimates of developmental functioning, Bagnato and Neisworth (1991) advocated for using a multifaceted, ecological approach to examine preschoolers’ developmental skills within social, school, and family contexts using multiple methods, traits, and sources. Two promising methods for use in ecological assessments are play-based assessment and semi-structured interviews. In addition to serving as an important context during which children learn and develop, play provides an important window through which to view development (Pellegrini, 1998). Play-based assessment provides information on developmental skills, reveals children’s patterns of interactions with caregivers and peers, and focuses on play as a primary context for learning and development (Paget, 1999). Play-based assessment is also functional, because the results can be linked immediately to intervention strategies (Lowenthal, 1997). Because play proceeds through a regular developmental sequence during childhood, evaluations of children’s play can provide indications of maturation and social competence. Some investigators have examined play within the context of parent-child interactions and peer interactions, while others have developed standardized measures of play to use in the assessment of children’s cognitive and language skills (e.g., Fewell, 1986; Linder, 1993; Westby, 1980). The present study used a measure of peer interactive play as an indicator of social competence. The Penn Interactive Peer Play Scale (PIPPS; Fantuzzo, Coolahan, Mendez, McDermott, & Sutton-Smith, 1998; Fantuzzo, Mendez, & Tighe, 1998) was developed in response to the paucity of psychometrically sound, developmentally appropriate, and culturally sensitive behavior rating scales for preschoolers. The PIPPS utilizes parent and teacher ratings of skills observed during play to distinguish children who demonstrate effective peer interactions from those who display less positive behaviors. Semi-structured interviews are also a viable method to gather parents’ and teachers’ perspectives about preschoolers’ social-emotional functioning. Advantages of an interview format include the opportunity to establish rapport, engage in discussion, and obtain a more elaborate description of children’s behaviors. The Vineland SocialEmotional Early Childhood Scales (SEEC; Sparrow, Balla, & Cicchetti, 1998) uses the semi-structured interview format, relying on caregiver responses to provide an overview of the child’s social and emotional functioning in the areas of attentional skills, social interactions, construction and observation of relationships, self-regulation, and understanding emotions. Parent-Teacher Convergence on Measures of Social-Emotional and Behavioral Functioning Although gathering information from multiple sources is critical in conducting ecological and comprehensive assessments (Neisworth & Bagnato, 2005), different raters often present evaluators with contradictory findings. Studies of behavior rating scales with school-age children have found wide variability in interrater reliability score coefficients and a consistent tendency for these to be lower than test-retest and internal consistency score reliability coefficients (Achenbach, McConaughy, & Howell, 1987; Elliott, Busse, & Gresham, 1993). Different status informants, such as parents and teachers, tend to give discrepant ratings of behavioral and emotional functioning, with a meta-analysis revealing a mean reliability coefficient of .28 (Achenbach et al., 1987). Score reliability coefficients generally are more consistent when informants are rating externalizing, as opposed to internalizing, behaviors (Kolko & Kazdin, 1993; Nickerson & Nagle, 2001) and correlations tend to be higher for children ages 6 through 11 than for adolescent samples (Achenbach et al., 1987). Although the majority of research in this area has been conducted with school-age children and adolescents, studies focused on preschoolers have found similar trends. Walker and Bracken (1996) investigated the consistency between parental reports of preschool behaviors in a sample of 64 parents (32 pairs) of 4and 5-year-old children, the majority of whom were Caucasian. Their results revealed large variations in consistency between mothers and fathers on four preschool behavior rating scales (r ranging from .09 to .79). Higher levels of consistency were found for clinical symptoms on behavior rating scales than for indicators of child temperament on a temperament scale. The consistency between mother and father ratings also was stronger when rating positive rather than negative behaviors. In their sample of 134 preschoolers (mean age = 57.4 months) of ethnic minority backgrounds attending pre-kindergarten Head Start programs, Fagan and Fantuzzo (1999) found statistically significant but relatively modest correlations (ranging from .17 to .54) for 6 of the 16 correlations between mothers’ and fathers’ ratings of social skills. In contrast, no statistically significant correlations emerged between mothers’ and teachers’ ratings and although 6 of the 20 correlations between fathers’ and teachers’ ratings were statistically significant, the magnitude of these correlations was only .17 to .25. Overall, these findings demonstrated a relative lack of consistency between parents’ and teachers’ ratings of preschoolers’ social skills. Findings of low convergence between parents and preschool teachers also have been found on ratings of temperament (Northam, Prior, Sansom, & Oberklaid, 1987) and behavior problems (Hinshaw, Han, Erhardt, & Huber, 1992). Northam et al. compared mothers’ and child care providers’ ratings of temperament in a sample of 43 children between 12 and 23 months and 60 children ages 24 to 39 months. One of the nine correlations yielded in the analyses with the younger group of children was statistically significant and moderate in strength (r = .49, p , .01), while three moderately strong, statistically significant correlations (r = .36 to .48) emerged for the older group. In the Hinshaw et al. study, parent and teacher ratings of 45 children, primarily of Caucasian descent and ranging in age from 3.5 to 6.8 years, essentially were uncorrelated for internalizing behaviors (r = .13) and modestly associated for externalizing behaviors (r = .32). Several possible reasons might account for the low convergence between parents’ and teachers’ ratings. Situation specificity is a commonly cited explanation that focuses on contextual issues, based on the assumption that behavior varies as a function of the setting in which it occurs (Kazdin, 1979). In addition, parents and teachers have different opportunities to observe behavior, with parents tending to have greater familiarity with their children’s pattern of speech and nonverbal cues (Diamond & Squires, 1993; Kaufman, Swan, & Wood, 1980). Rater biases of informants, including response set, social desirability, and frame of reference, also have been found to affect convergence (Nickerson & Nagle, 2001; Youngstrom, Loeber, & Stouthamer-Loeber, 2000). Child variables, such as socio-economic and racial-ethnic differences, also have been correlated with low rater convergence (Fagan & Fantuzzo, 1999; Kaufman et al., 1980; Youngstrom et al., 2000). For preschool children, differences in ratings also might be due to transitional periods in development, when emerging competencies are not yet displayed consistently across settings (Diamond & Squires, 1993). Existing research frequently has conceptualized inter-rater reliability from a unitary perspective by focusing primarily on the consistency of two ratings by calculating correlation coefficients. Stemler (2004) asserted that this common practice of describing inter-rater reliability as a single, unitary concept is imprecise. Instead, he differentiated between three different categories of inter-rater reliability: (a) consensus estimates, which measure agreement; (b) consistency estimates; and (c) measurement estimates, which assess the extent to which raters assess underlying common constructs. The large majority of published studies have used behavior rating scales to assess convergence between parents’ and professionals’ perspectives about children’s social and behavioral competence; however, there is evidence that convergence varies based on the content assessed (e.g., specific skills vs. global performance) and the format of the assessment (e.g., recognition vs. recall; Diamond & Squires, 1993). Play-based assessment and semi-structured interviews of preschool social-emotional functioning, therefore, might yield different findings of rater consistency as compared to previous results lacking convergence. Szatmari, Archer, Fisman, and Streiner (1994) found consistency between parent and teacher ratings of 83 children ages 4 to 6 years with developmental delays (r = .60) on the socialization domain of the Vineland Adaptive Behavior Scales. In terms of peer interactive play, Fantuzzo et al. (1998) found statistically significant but moderately low (r = .18 to .36) correlations between corresponding domains on the parent and teacher versions of the PIPPS in a sample of 297 primarily African American children ages 37 to 64 months attending an urban Head Start program. Taken together, these findings support the premise that the type and content of assessment tools might influence rater convergence positively, but further study is needed. The purpose of the present study was to examine three aspects of convergence between parent and teacher ratings on two different measures of behavioral and social-emotional functioning in a sample of preschool children at risk for academic difficulties. Specific research questions were: (a) Is there agreement (consensus) between parent and teacher ratings on the PIPPS and SEEC? (b) How consistent are parent and teacher ratings on the PIPPS and SEEC? and (c) Do parent and teacher ratings on the PIPPS and SEEC reflect measurement of common underlying constructs?